Dual arm robot

ABSTRACT

The dual arm robot includes a first arm including a first hand, a first visual sensor and a first force sensor, and a second arm including a second hand, a second visual sensor and a second force sensor, uses each visual sensor to detect positions of a lens barrel and a fixed barrel to hold and convey them to a central assembling area, uses the first visual sensor to measure a position of a flexible printed circuits to insert the flexible printed circuits into the fixed barrel, and uses outputs of the force sensors to fit and assemble the fixed barrel onto the lens barrel under force control. The dual arm robot converts a position coordinate of a workpiece detected by each visual sensor to a robot coordinate to calculate a trajectory of each hand and drive each arm, to thereby realize cooperative operation of the two arms.

TECHNICAL FIELD

The present invention relates to a dual arm robot having a structure, inwhich two arms, each being provided with a visual sensor, arecooperatively operated with each other to assemble workpieces.

BACKGROUND ART

Assembling work by a robot has been widely adopted, and most robots havea position and posture instructed in advance and perform repetitive workaccording to the instruction. In recent years, there has been a demandfor more advanced work such as fitting one part to another. As disclosedin International Publication NO. WO98/017444, there is known a forcecontrol robot system which uses workpiece position detection by a visualsensor and force control by a force sensor. This system includes, asillustrated in FIG. 4, a force sensor 103 and a hand 104 mounted on afinger section 102 of a robot 101, a controller 105 for controlling therobot 101, etc., and fits a first part 107 held by a holding claw of ahand 104 into a second part 108. The second part 108 is positioned by apositioning device 109, and a visual sensor 110 for detecting thepositions and postures of the first part 107 and the second part 108 isfixed to a surface plate 111.

The force control robot system includes a hand for holding a first part,a robot including a force sensor for detecting force applied to thefirst part, a visual sensor for obtaining image data to obtain apositional relationship between the first part and a second part, and acontrol unit for controlling the robot and the visual sensor. Thecontrol unit includes a fitting operation performing unit for moving thefirst part held by the hand of the robot close to the second part andperforming fitting operation under force control based on an output ofthe force sensor. The control unit also includes a correcting unit forobtaining workpiece position data representing a positional relationshipbetween the first part and the second part based on image data obtainedby the visual sensor, and for correcting the position and posture of therobot based on the obtained workpiece position data, in advance of thefitting operation. Further, the control unit includes a discriminatingunit for obtaining data representing a fitting state of the first partin the second part based on image data of the first part and the secondpart obtained by the visual sensor, and for discriminating whether ornot the parts are in a properly fitted state based on the obtainedfitting state data, after completing the fitting operation.

Dual arm robots with two arms are also beginning to be introduced toproduction sites. As disclosed in Japanese Patent Application Laid-OpenNo. 2008-168406, a workpiece mounting system is known in which a dualarm robot can hold two workpieces and mount one workpiece on anotherwith high precision.

SUMMARY OF INVENTION

In the conventional examples described above, there are the followingproblems due to the visual sensor being fixed at one place and always inthe same position.

If one workpiece has a moving portion like a hinged fold-open/closeportion or a cable, it is difficult to perform complicated work asassembling one workpiece to the other workpiece while lifting anddeflecting the moving portion. In other words, it is difficult tocooperatively operate two arms in such a manner that the two arms eachrecognize the position of the workpiece held by the other andsimultaneously operate to assemble the workpieces, including theoperation of assembling one workpiece to the other workpiece whilemanipulating the posture of the moving portion.

In assembling a flexible object including flexible printed circuits(flexible printed board) and a cable, even after the flexible object isheld by the hand, the flexible object is bent under the influence ofgravity and inertia force during conveyance, and therefore the shape ofthe flexible object is changed over time. In order to achieve highassembling precision, posture change of the flexible object is requiredto be monitored by a visual sensor. However, it is difficult for astationary visual sensor to track posture change of a flexible object.

In order to detect a predetermined portion of a workpiece by a visualsensor, a jig for clamping a workpiece in place is required. As aresult, when a type of workpiece is changed, it is necessary to make asetup change to modify the jig, which leads to a time consuming andcostly process.

The present invention provides a dual arm robot, which has broadutility, and is able to perform complicated work efficiently andcooperatively.

In order to achieve the above-mentioned objects, according to thepresent invention, a dual arm robot causing a first arm and a second armto cooperatively operate, includes: the first arm including a first handand a first visual sensor; the second arm including a second hand and asecond visual sensor; and a controller which uses an output of one ofthe first visual sensor and the second visual sensor, to thereby controlone of the first arm and the second arm including another of the firstvisual sensor and the second visual sensor.

In the configuration described above, the two arms each can move theirvisual sensors close to each workpiece to detect a position coordinateof the workpiece. The detected position coordinate of the workpiece isconverted from a visual sensor coordinate system to a robot coordinatesystem, a trajectory required to hold the workpiece is calculated withreference to the robot coordinate system, and the workpiece is held byeach hand. The position coordinate of the workpiece held by each hand isdetected again by the visual sensors, to thereby confirm a relativeposition between the first arm and the second arm. Next, a trajectory ofeach hand required for assembly work is calculated to control a movementof each arm. Each arm is provided with a visual sensor, and an output ofone visual sensor is used to control the arm equipped with the othervisual sensor. Therefore, two arms share and recognize the positioncoordinate of the workpiece to perform assembly work.

In assembling workpieces including a flexible object such as flexibleprinted circuits and a cable, it is possible to achieve high assemblingprecision by using the visual sensor to track the flexible object whichchanges its shape over time.

It is possible to keep the posture of a workpiece by the hand of one armwhile detecting the position coordinate of the workpiece by the visualsensor of the other arm, to thereby perform assembly work. Therefore, itis not necessary to change the jig for fixing the workpiece every time atype of workpiece is changed, and it is also not necessary to make asetup change to modify the jig, as a result of which the time and costare reduced.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a perspective view generally illustrating a dual arm robotaccording to an embodiment.

FIG. 1B is a view illustrating a coordinate conversion of the dual armrobot according to the embodiment.

FIG. 1C is a block diagram illustrating a configuration of the dual armrobot according to the embodiment.

FIG. 2A is a flowchart illustrating a work sequence according to theembodiment. FIG. 2 is comprised of FIGS. 2A and 2B.

FIG. 2B is a flowchart illustrating a work sequence according to theembodiment. FIG. 2 is comprised of FIGS. 2A and 2B.

FIG. 3A illustrates an operation of the apparatus illustrated in FIGS.1A, 1B and 1C, in a situation in which flexible printed circuits arebeing aligned with a fixed barrel by cooperative operation.

FIG. 3B illustrates an operation of the apparatus illustrated in FIGS.1A, 1B and 1C, in a situation in which a hand is being retracted whilethe flexible printed circuits is inserted into the fixed barrel.

FIG. 4 is a view illustrating a related example.

DESCRIPTION OF EMBODIMENT

As illustrated in FIG. 1A, this embodiment is a dual arm robot whichperforms assembly work of a lens barrel of a camera. A first arm 1includes a first hand 2 and a first visual sensor 3 fixed to a wristsection of the first arm 1, the wrist section having a first forcesensor 4 mounted thereon (incorporated therein). A second arm 5 includesa second hand 6 and a second visual sensor 7, and a wrist section of thesecond arm 5 has a second force sensor 8 mounted thereon (incorporatedtherein). The two arms 1 and 5 each include a control unit forcontrolling a movement of the arm, and are mutually in communicationwith a controller (now shown).

Output value data (output) of each visual sensor for detecting aposition (position coordinate) of each hand and each force sensor areall commonly used by the controller (control unit). By performingcoordinate conversion and calculation of a trajectory for assembly workbased on the data, cooperative control required for assembly work of alens barrel 9 and a fixed barrel 10 as workpieces is performed. Thefixed barrels 10 are placed on a fixed barrel supply tray 11. The lensbarrels 9 are placed on a lens barrel supply area 12, the lens barrels 9each including flexible and thin flexible printed circuits 9 a whichextend from a top end of the lens barrel 9. In an assembling area 13,the lens barrel 9 and the fixed barrel 10 are assembled together. Anassembled lens barrel 14 is placed in an assembled lens barrel storagearea 15.

FIG. 1B illustrates a relationship among a workpiece coordinate Pw whichis a position coordinate of a workpiece in a process of holding the lensbarrel 9 using the second arm 5, the second hand 6, and the secondvisual sensor 7, a visual sensor coordinate system Σv, and a robotcoordinate system Σr which represents the position coordinate of thesecond hand 6. This also applies to the first arm 1, the first hand 2,and the first visual sensor 3. The visual sensor coordinate system Σv isa coordinate system with the origin at the position of the providedvisual sensor. The robot coordinate system Σr is a coordinate systemwith the origin at the base of the arm.

Those coordinate systems are each set in the controller (control unit).

FIG. 1C is a block diagram illustrating a configuration of the dual armrobot according to this embodiment. A first robot and a second robot areeach mutually in communication with the controller.

The output values of the force sensors are stored in a force sensoroutput value storage section, and based on the obtained results, atrajectory calculating unit corrects the movements of the arms and thehands.

The output of the visual sensor mounted on the first robot and theoutput of the visual sensor mounted on the second robot both areprocessed by an image processing unit, and the shape, position orposture of a workpiece is extracted. A workpiece position coordinatedetecting unit calculates the position of the workpiece using theprocessed data.

Here, by using the data of workpiece shapes stored in a workpiece shapedata storage section in advance, the workpiece position coordinatedetecting unit can detect the position and posture of a workpiece withhigher precision.

A robot position coordinate storage section stores a work sequencetaught in advance and the position coordinates for the movements of thearms and the hands. The trajectory calculating unit performs correctionfrom the outputs of the force sensors and the visual sensors, to therebycorrect the movements of the arms and the hands. The dual arm robotincludes a control unit to operate each of the arms and the hands basedon the results of trajectory calculation.

In the workpiece position coordinate detecting unit, it is determinedwhether or not the arm equipped with one visual sensor is controlled byusing an output of the other visual sensor.

From an operational flowchart set for the dual arm robot, it may beknown in advance that a workpiece is visible from one visual sensor butnot from the other visual sensor in a particular operational step. Inthat case, for the particular operational step, the workpiece positioncoordinate detecting unit may detect the position/posture of theworkpiece based on an output of one visual sensor, and the trajectorycalculating unit may calculate the trajectory of the arm equipped withthe other visual sensor.

In detecting the position/posture of a workpiece by means of, forexample, template matching with respect to the data of the workpieceshapes described above, image information from one visual sensor alonemay not be sufficient to recognize the workpiece. The workpiece positioncoordinate detecting unit may be configured to use an output of theother visual sensor to deal with this situation. In this case, thetrajectory calculating unit may be configured to perform a trajectorycalculation according to the output of the other visual sensor if theoutput of the other visual sensor is used.

The arms are each provided with a visual sensor, and the outputs of bothvisual sensors are processed by the image processing unit. Therefore,two arms are capable of sharing and recognizing the position coordinateof the workpiece to perform assembly work.

In the following description, the assembly work of the lens barrel isdescribed by way of example and with reference to FIGS. 1A to 1C, 2A,2B, 3A, and 3B.

In the cooperative operation illustrated in the flowchart in FIGS. 2Aand 2B, as illustrated in FIG. 3A, the first arm 1 and the second arm 5move the flexible printed circuits 9 a of the lens barrel 9 and thefixed barrel 10 close to each other while aligning the direction of thetop end of the flexible printed circuits 9 a with the direction of thecentral axis of the fixed barrel 10. Then, as illustrated in FIG. 3B,after inserting the flexible printed circuits 9 a into the fixed barrel10, the first hand 2 is retracted while the second hand 6 is lowered,and the flexible printed circuits 9 a are inserted into the fixed barrel10.

The work sequence of this embodiment is described according to theflowchart in FIGS. 2A and 2B. First, the first arm 1 moves near thefixed barrel supply tray 11 and is oriented to detect the position ofthe fixed barrel 10 arranged on the fixed barrel supply tray 11 by thefirst visual sensor 3. The second arm 5 moves near the lens barrelsupply area 12 and is oriented to detect the position of the lens barrel9 by the second visual sensor 7 (S1).

The first arm 1 uses the first visual sensor 3 to detect a holdingposition of the fixed barrel 10. Shape data of the workpiece created bya system such as computer aided design (CAD) is stored in the controllerin advance. In detecting an edge of the fixed barrel 10 using the firstvisual sensor 3, the workpiece position coordinate detecting unit refersto the shape data to calculate a position of a center point of a circlerepresenting the edge as the workpiece coordinate. From the relativedistance between the workpiece coordinate and the hands, the trajectorycalculating unit calculates distances the arms are to move, and each armmoves to its holding position.

To control the movement of each arm, a conversion unit included in thecontroller performs coordinate conversion of the workpiece coordinate,and converts the visual sensor coordinate system to the robot coordinatesystem. The conversion unit performs coordinate conversion with respectto a fixed barrel coordinate which is the workpiece coordinate of thefixed barrel 10 and a lens barrel coordinate which is the workpiececoordinate of the lens barrel 9, respectively. Similarly, the second arm5 uses the second visual sensor 7 to detect the position of the lensbarrel 9, and the second arm 5 moves to its holding position (S2).

The holding positions of the workpieces are stored in the controller inadvance, and the controller calculates the holding positions based onthe workpiece coordinates detected in Step S2. The first hand 2 holdsthe fixed barrel 10 and the second hand 6 holds the lens barrel 9 (S3).

The first arm 1 waits, while holding the fixed barrel 10 above theassembling area 13. The second arm 5 places the lens barrel 9 on theassembling area surface. The second arm 5 lowers the lens barrel 9,while keeping the posture of the lens barrel 9 held by the second arm 5perpendicular to the assembling area surface. The second arm 5 reducesthe lowering speed as a bottom surface of the lens barrel 9 and theassembling area surface get closer, and contacts the bottom surface ofthe lens barrel 9 with the assembling area surface. The distance betweenthe bottom surface of the lens barrel 9 and the assembling area surfaceis calculated from the moving distance of the second arm 5. When thebottom surface of the lens barrel 9 contacts with the assembling areasurface, the output value of the second force sensor 8 increases. Thecontroller detects that the lens barrel 9 is placed on the assemblingarea surface when the output value exceeds a threshold value and causesthe second hand 6 to release its hold of the lens barrel 9 (S4).

The first arm 1 keeps the posture of the fixed barrel 10 in a positionwhere the fixed barrel 10 is easily held by the second arm 5. The secondarm 5 uses the second visual sensor 7 to detect the position of thefixed barrel 10 held by the first hand 2 (S5).

The second hand 6 holds the fixed barrel 10 held by the first hand 1,which is detected in Step S5. After the fixed barrel 10 is held by thesecond hand 6, the first hand 2 releases its hold of the fixed barrel10. The controller monitors output values of the first force sensor 4and the second force sensor 8 during the handover of the fixed barrel10, to thereby perform force control so as to prevent the fixed barrel10 from being broken due to the application of excessive force (S6).

The placement position of the lens barrel 9 is already known as theposition of the lens barrel 9 placed by the second arm 5 in Step S4. Thecontroller estimates the posture of the lens barrel 9 and the positionof the flexible printed circuits 9 a based on the placement position,and picks up an image of the lens barrel 9 using the first visual sensor3. The flexible printed circuits 9 a tend to change their posturebecause the flexible printed circuits 9 a are thin and flexible.Therefore, if the first visual sensor 3 does not succeed to pick up animage which is sufficient to detect the position of the flexible printedcircuits 9 a with high precision in the first attempt, the controllerestimates the front position of the flexible printed circuits 9 a basedon the detected edge. The controller changes the image pick-up angle sothat the flexible printed circuits 9 a face the front, and picks up animage again to detect the top end of the flexible printed circuits 9 awith high precision. At the same time, the position of the fixed barrel10 held by the second hand 6 is detected by the second visual sensor 7(S7).

Next, the first hand 2 holds the flexible printed circuits 9 a. Thesecond arm 5 moves the fixed barrel 10 held by the second arm 5 towardthe top of the lens barrel 9 (S8).

The flexible printed circuits 9 a held by the first hand 2 are deformeddue to the holding action, and hence the first visual sensor 3 cannotpick up a front image of the flexible printed circuits 9 a. Therefore,the second visual sensor 7 picks up the front image of the flexibleprinted circuits 9 a to detect the position of the flexible printedcircuits 9 a. From the position of the fixed barrel 10 and the positionof the flexible printed circuits 9 a detected in Step S7, the trajectorycalculating unit of the controller calculates a trajectory in which thedirection of the central axis of the fixed barrel 10 is aligned with thedirection of the top end of the flexible printed circuits 9 a and theflexible printed circuits 9 a extend through the fixed barrel 10. Thecontroller operates the first arm 1 and the second arm 5 simultaneouslyaccording to the calculated trajectory. As illustrated in FIG. 3A, thefirst arm 1 and the second arm 5 move the flexible printed circuits 9 aand the fixed barrel 10 close to each other, while aligning thedirection of the top end of the flexible printed circuits 9 a with thedirection of the central axis of the fixed barrel 10, to thereby insertthe top end of the flexible printed circuits 9 a into the center of thefixed barrel 10 (S9).

Next, the first hand 2 releases its hold of the flexible printedcircuits 9 a, and the first hand 2 retracts from between the fixedbarrel 10 and the lens barrel 9. At the same time of the retraction, theflexible printed circuits 9 a tend to return to their original shape bytheir resiliency and gravity and get out of the fixed barrel 10. Inorder to prevent the flexible printed circuits 9 a from getting out ofthe fixed barrel 10 concurrently with the retraction of the first hand2, the second hand 6 keeps the top end of the flexible printed circuits9 a and the central axis of the fixed barrel 10 in alignment as thesecond hand 6 moves downward, and inserts the flexible printed circuits9 a into the fixed barrel 10 as illustrated in FIG. 3B (S10).

The retracted first hand 2 holds the lens barrel 9 and keeps itsposture. Next, the first visual sensor 3 detects the position of thecentral axis of the lens barrel 9, and the second visual sensor 7detects the central axis of the fixed barrel 10. The first arm 1 and thesecond arm 5 assemble the fixed barrel 10 and the lens barrel 9 in sucha manner that their central axes align with each other. Here, the firstarm 1 measures force applied to the lens barrel 9 from the output valueof the first force sensor 4, and the second arm 5 measures force appliedto the fixed barrel 10 from the output value of the second force sensor8. The output value of each force sensor can be used for correcting theassembly trajectory, to thereby fit the fixed barrel 10 onto the lensbarrel 9 under force control (S11).

When the output value from the second force sensor 8 exceeds thethreshold value, the fitting and assembling is completed. The firstvisual sensor 3 is used to pick up a side image of the lens barrel 9 toconfirm the completion of the fitting and assembling. The assembled lensbarrel 14 is conveyed to the assembled lens barrel storage area 15 bythe second arm 5 (S12). After the assembled lens barrel 14 is conveyed,the process returns to Step S1 and each arm moves to the workpiecesupply position to start the next assembly work. The first arm 1 and thesecond arm 5 are cooperatively operated with each other from Step S7 toStep S11.

The first arm 1 and the second arm 5 move the flexible printed circuits9 a and the fixed barrel 10 close to each other, while aligning thedirection of the top end of the flexible printed circuits 9 a with thedirection of the central axis of the fixed barrel 10, and insert theflexible printed circuits 9 a into the fixed barrel 10. Subsequently,the second hand 6 moves downward concurrently with the retraction of thefirst hand 2, to thereby insert flexible printed circuits 9 a into thefixed barrel 10. The flexible printed circuits 9 a, which are thin andflexible, are easy to deform. Therefore, although it has been difficultto perform assembly work by a robot, the assembly work can be realizedby the cooperative operation of the dual arm robot according to thisembodiment.

After the flexible printed circuits 9 a are held by the first hand 2,even if its posture is changed due to the holding action and the firstvisual sensor 3 cannot pick up the front image of the flexible printedcircuits 9 a, it is possible to use the second visual sensor 7 to pickup the front image of the flexible printed circuits 9 a to detect itsposition with high precision.

Even if a workpiece is in a position or posture which is difficult to bedetected by one visual sensor, it is possible to operate the othervisual sensor to pick up the image of the workpiece to detect itsposition with high precision. As a result, highly precise assembly ispossible.

The first visual sensor 3 detects the position of the central axis ofthe lens barrel 9, and the second visual sensor 7 detects the centralaxis of the fixed barrel 10. The first arm 1 and the second arm 5assemble the fixed barrel 10 and lens barrel 9 in such a manner thattheir central axes align with each other. Here, the first arm 1 measuresforce applied to the lens barrel 9 from the output value of the firstforce sensor 4, and the second arm 5 measures force applied to the fixedbarrel 10 from the output value of the second force sensor 8. The outputvalue of each force sensor can be used to correct the assemblytrajectory of each arm, to thereby fit the fixed barrel 10 onto the lensbarrel 9 under force control. As a result, it is possible to realizehighly precise fitting of workpieces without a jig for clamping aworkpiece.

Note that, although the visual sensor is configured to be provided inthe wrist section of the arm in this embodiment, it can be mounted onthe hand section with the same effect. Further, although the output ofthe second visual sensor is used to perform the trajectory calculationof the first arm by way of example in this embodiment, the first visualsensor may be used for the second arm in the same manner.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2009-274307, filed Dec. 2, 2009, which is hereby incorporated byreference herein in its entirety.

The invention claimed is:
 1. A dual arm robot causing a first arm and asecond arm to cooperatively operate, comprising: the first arm includinga first hand and a first visual sensor; the second arm including asecond hand and a second visual sensor; and a controller, wherein, in anoperation for assembling a first workpiece and a second workpiece byusing the first arm and the second arm, when the first visual sensor isable to detect the first workpiece and the second visual sensor is ableto detect the second workpiece, the first visual sensor detects thefirst workpiece held by the first arm and the second visual sensordetects the second workpiece held by the second arm, wherein, in theoperation, when the first visual sensor is unable to detect the firstworkpiece, the second visual sensor detects a position coordinate of thefirst workpiece, or when the second visual sensor is unable to detectthe second workpiece, the first visual sensor detects a positioncoordinate of the second workpiece, and wherein the controller uses arobot coordinate system containing position coordinates of the first andsecond hands, a first visual sensor coordinate system containing theposition coordinate of a workpiece detected by the first visual sensor,and a second visual sensor coordinate system containing the positioncoordinate of a workpiece detected by the second visual sensor, so thatthe controller converts the position coordinate of the workpiece fromthe first and second visual sensor coordinate systems to the robotcoordinate system and controls movements of the first and second arms inthe operation for assembling the first workpiece and the secondworkpiece.
 2. A dual arm robot according to claim 1, further comprising:a first force sensor mounted on the first arm; and a second force sensormounted on the second arm, wherein the controller corrects the movementsof the first and second arms based on output of the first and secondvisual sensors.